A Principled Method of Scenario Design for Testing Emergency Response Decision-Making

We are investigating decision aids that present potential courses of action available to emergency responders. To determine whether these aids improve decision quality, however, we first developed test scenarios that were challenging in well-understood ways to ensure testing under the full breadth of representative decision-making situations. We devised a three-step method of developing scenarios: define the decision space, determine the cost components of each decision’s potential consequences based on the principles of Robust Decision Making, then choose conflicting pairs of cost components (e.g., a small fire, implying low property damage, in a densely inhabited area, which implies high personal injury). In a validation of this approach, experiment participants made decisions faster in non-ambiguous cases versus cases that included this principled introduction of ambiguity. Our Principled Ambiguity Method of scenario design is also appropriate for other domains as long as they can be analyzed in terms of costs of decision alternatives.